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1.  INTRODUCTION 


The  derivative  of  a  function  can  be  defined  as  the  limit  of  a  sequence  of  divided 
differences.  Consequently,  it  is  independent  of  any  finite  number  of  values  of  this 
sequence,  i.e.,  it  cannot  be  obtained  from  a  finite  set  of  observed  functional  values  without 
additional  assumptions.  For  instance,  suppose  we  compute 


and  obtain  =  0  for  =  1,2,...,  n  for  an  arbitrary  but  finite  n.  Can  we  conclude  that 
yXO)  -  0?  Certainly  not,  since  y(t)  -  sin  with  integer  m  >  n  would  also  yield  R{k)  = 

0  for  k  —  1,2,  ...,  n .  Yet  we  have  y'CO)  —  2"*^  which  we  can  make  as  large  as  we  please  by 
selecting  a  sufficiently  large  integer  m. 

However,  we  can  approximate  the  derivative  of  a  tabular  function  if  we  make 
certain  assumptions  about  the  derivative  or  the  tabular  function.  Different  assumptions 
lead  to  a^ variety  of  numerical  methods.  Until  the  1960’s  little  research  was  done  on  this 
problem.  However.during  the  last  two  decades  quite  a  few  research  papers  on  the  subject 
have  been  published.  The  purpose  of  this  report  is  to  discuss  certain  classical  and  recently 
developed  methods  for  estimating  derivatives  from  discrete  data.  The  methods  range 
from  divided  differences,  through  smoothing  by  splines  and  by  regularization  procedures, 
to  the  autoregressive  approach.  Another  purpose  is  to  show  that  these  methods  can  be 
arranged  in  the  order  of  subsequent  refinements  or  generalizations,  although  most  of  them 
were  developed  from  independently  selected  assumptions.  Such  an  arrangement  can  be 
us^  as  an  indication  of  improved  ability  to  approximate  unknown  derivatives  at  the  cost 
of  increasing  complexity  of  the  method. 

It  should,  though,  be  emphasized  that  no  method  can  be  best  in  every  case  For 
instance,  suppose  we  observe  two  functions  y^U)  =  0  and  -  sin  nOirt  at  the  points 
t„  -0.2«,n>^),l,2,...,6  and  that  our  observations  are  exact.  Then  the  observed  values  are 
all  zeros  in  both  cases.  Hence,  even  though  a  method  M  produces  good  values  of  y'(t„), 
i.e.,  all  close  to  zero,  it  will  give  inadequate  estimates  of  y^iO  all  of  which  are  equafto 

120  TT. 


^  E.  Issacson  and  H.  B.  Keller,  Analysis  of  Numerical  Methods.  John  Wiley.  New  York  (1966) 
pp  288-294. 
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This  example  shows  that  a  good  method  should  also  provide  information  about  the 
trequency  components  whose  contributions  to  the  derivative  cannot  be  detected  from  the 
(^ta.  The  spectral  analysis  method  and  autoregressive  approach^  provide  this  explicitly 
the  fost  by  deriving  the  spectrum  of  the  underlying  function,  the  second  by  determining 
all  the  frequencies  com^tible  with  the  data  and  by  indicating  those  for  which  the 
amplitudes  can  be  determined  from  the  data  with  sufficient  accuracy. 

Section  2  of  this  report  begins  with  a  discussion  of  classical  methods  of  divided 
differences,  polynomial  interpolation,  and  regression  methods,  including  their  refinement 
by  application  of  splines.  Section  3  shows  that  the  regularization  procedure^’^  can  be 
inteyrejed  as  a  refinement  of  the  spline  approach.  Section  4  shows  that  the  autoregressive 
method  is  a  refinement  of  the  regularization  procedure. 

The  numerical  method  for  computing  the  derivative  of  an  analytic  function  by  J.  N. 
Lyness  and  C.  B.  Moler  is  not  discussed  here  since  it  requires  evaluation  of  the  function 
for  complex  values  of  its  argument.  Although  frequently  an  observed  function  may  be 
assumed  to  be  analytic,  its  continuation  into  a  complex  domain  from  a  finite  set  of 
observed  values  in  a  real  domain  is  even  more  difficult  than  evaluation  of  its  derivative. 

hus,  this  method  cannot  be  applied  to  evaluate  a  derivative  from  a  finite  set  of  observed 
values  corresponding  to  real  values  of  the  indepiendent  variable. 


R.  S.  Anderssen  and  P.  Bloomfield,  ”A  Time  Series  Approach  to  Numerical  Differentiation.” 
Technometrics.  16  (l)( 1974),  pp  69-75. 

^  V 

C.  Masaitis  and  G.  Francis.  "Numerical  Differentiation  of  Noisy  Data,”  BRL  Memorandum 
Report  3126,  Aug  81  (AD  A104631). 


4 

Pickonov,  Solution  of  Incorrectly  Formulated  Problems  and  the 
Method."  Soviet  Math.  Dokl.  4  (1963),  pp  1035-1038. 


Regularization 


J.  Cullum,  Numerical  Differentiation  and  Regularization,”  SIAM  J.  Numer.  Anal.  8  (1971). 
pp  254-265.  ~~  ^ 


J.  N  Lyness  and  C.  B.  Moler,  "Numerical  Differentiation  of  Analytic  Functions,”  SIAM  J 
Numer.  Anal  4  (1966),  pp  202-210.  - ^ 
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2.  POLYNOMIAL  AND  REGRESSION  METHODS 

Let  x„  be  an  observed  value  of  an  unknown  function  yit)  at  the  point  t  -  nh  for 

some  A  >  0  and  n=l,2,  N.  Assume  that  yithC^  and  that  the  error  of  observation  is 
6„,  i.e., 

x„-y(nh)  +  €„  ,  (2.1) 


with 


Un  I  <  8. 


(2.2) 


The  simplest  way  of  obtaining  an  approximation  '  of  the  derivative  y'inh )  is  by  the 
divided  difference: 


I  _  ('^+1  l) 

'  2A  ^ 


(2.3) 


i.e.,  in  view  of  (2.1) 


"  Th 


^n+l  ] 

2h 


(2.4) 


By  Taylor’s  formula  we  have: 


■^(^n+i)  “y(?„-i)  +  2hy'(T„), 


(2. .5, 


with  <  T„  < 

It  follows  from  (2.4)  and  (2.5)  that 


x„' -y'(T„)  + 


^n+l  ^n-l 

2h 


(2.6'> 


If  the  error  term  €„_{)/2h  is  negligible  we  get  from  (2.6):  x„'^.>-'(t„),  i.e.,  this 

approximation  equates  the  derivative  at  the  midpoint  between  ?„_i  and  to  its  Value  at  a 
certain  unknown  intermediate  ixjint  t„6/„  where  /„  is  the  interval  .  Thus,  we 
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(2.7) 


yXO  -X,'  -yxo  -y(T„)  -  , 

In 

or,  in  view  of  (2.2),  the  error  of  this  estimate  is  bounded  as  follows: 

+  1  .  (2.8) 


Again,  by  Taylor’s  formula  we  have 

y'(t„)-y'(T„)  +  (T„ -f„)3;"(T„')  ,  (2.9) 

•where  t„'  is  an  intermediate  value  between  t„  and  f„,  i.e.,  .  Obviously 

M/I  “^/i  I  ^  2/;.  Since  we  assume  that  is  continuous  it  is  bounded  by,  say,  on  the 
interval  Therefore  we  obtain  from  (2.8)  and  (2.9); 

2hM„  +  |-  .  (2.10) 

If  we  know  the  bound  M  ■^M„  of  the  second  derivative  in  the  interval  and  the 

bound  8  we  can  choose  the  step  size  h  equal  to,  say,  h„  which  minimizes  the  error  of  the 
estimate  (2.3).  The  value  of  hg  is  obtained  by  minimizing  the  right-hand  side  of  (2.10) 
with  M„  replaced  by  M.  Thus, 


h 


O 


(2.11) 


We  can  use  this  optimal  step  size  if  we  know  M  and  8  before  observations  are  made.  If 
we  have  to  differentiate  already  collected  data  then  we  can  use  (2.11)  provided  the  step 
size  of  the  observations  h  <.  hg.  In  this  case, we  use  the  data  with  the  step  size  h'  kh  with 
k  being  a  positive  integer  such  that  kh  is  as  close  as  possible  to  hg. 

This  choice  of  hg  minimizes  the  error  of  the  estimate  (2.3)  in  the  worst  case, 
namely  when  the  absolute  value  of  the  second  derivative  is  equal  to  its  upper  bound  and 
the  error  is  ^ual  to  its  bound  8.  In  a  practical  situation  such  a  case  may  never  occur  since 

the  observation  errors  may  be  relatively  small  where  the  second  derivative  is  exceptionally 
large. 


Furthermore,  it  may  be  much  more  important  to  have  an  average  error,  such  as  the 
root  mean  square  error  (RMSE)  small,  even  at  the  exp)ense  of  larger  errors  in  an  extreme 
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By  assuming  that  observation  errors  are  mutually  independent  and  have  zero  means 
and  (2^)^  ^  ^  stochastic  process  independent  of  observation  errors  we  get  from  (2.7) 


'•  ^  2h^ 

where  o-^  is  the  variance  of  e„ . 

Since  |t„  —t„\  ^2h  we  have 

Let  j  .  Then  the  expected  value  of  the  square 

minimized  by 


(2.13) 

of  the  estimated  error  is 


(2.14) 


Sup^se  now  that  the  observed  data  are  distance  traveled  by  a  vehicle  with  maximum 
acceleration  a  m/s  and  that  all  values  of  acceleration  are  equally  likely.  Then  =  1/2 
and,  thus,  by  (2.13)  and  (2.14)  the  error  bound  is  n  /  a 


E 


^  'J2a  <T  . 


(2.15) 


K  observation  errors  are  normally  distributed  and  that  we  make 

about  100  observations.  Then  a  reasonable  bound  of  the  error  is  8  =  3.5o-.  In  this  case  the 

optimal  value  of  h  given  by  (2.11)  is  h,  ~O.Scr/2ayl\  This  value  substituted  in  (7  13) 
yields  V- .  .  f 


TOa^ 
3.5 


(2.16) 


i.e.,  an  error  bound  20-percent  higher  than  that  in  (2.15).  This  simple  case  illustrates  that 

'he  best  approximation  in  terms 

or  the  RMS  or  some  other  average. 


9 


The  approximation  (2.3)  is  exact  for  the  polynomials  of  the  first  degree,  provided 
the  data  are  exach  A  straightforward  generalization  is  an  approximation  that  produces 
exact  denvattves  for  polynomials  of  degree  k  or  less,  assuming  exact  data.  oLouT 
differentiation  of  an  interpolating  polynomial  is  such  a  procedure.  An  interpolatinii 
polynomial  is  given  by  either  Lagrange’s,  Newton’s,  Gauss’s  or  any  other  pol^omW 
interpolation  formula.  Instead  of  differentiating  an  interpolating  polynomial  we  caLbtain 
e  same  estimate  by  an  iterative  procedure  such  as  that  described  by  D  B  Hunter'  or  H 
C.  Hershey  et  al.  ' 

AccoMing  to  Weierstrass’  theorem,  a  continuous  function  can  be  approximated 
arbitrarily  closely  by  polynomials  of  increasing  degree.  However,  approximations  of 
erivatives  obtained  by  differentiating  these  polynomials  may  be  very  poor.  If  the  data  are 
interpolated  by  Bernstein  polynomials  of  increasing  degree  k  then  a  uniform 
approximation  of  the  function  by  B^.  and  of  its  derivative  by  B^'  is  obtained.^  However 
even  small  observation  errors  may  affect  the  derivatives  B^'  drastically  and,  thus  caus^ 
arge  errors  in  the  estimates  of  y'it).  Besides,  round-off  errors  in  computing  coefficients  of 
these  polynomials  for  large  values  of  k  are  very  substantial.  Therefore,  in  practice  the 

degree  of  an  interpolating  polynomial  is  chosen  equal  two  or  three.  Even  in  this  cas^  the 
resulting  errors  may  be  very  large. 

differentiating  interpolating  polynomials,  derivatives  are 
estimated  by  differentiating  a  low  degree  polynomial  that  is  obtained  by  a  least  squares 
approximation  of  a  sequence  of  data  points  located  symmetrically  on  both  sides  of  the 
point  where  the  value  of  derivative  is  estimated.^®  This  is  a  common  procedure  of  moving 
polynomial  arcs  and  it  provides  a  degree  of  smoothing  of  the  observation  errors  by 
averaging  their  effects  over  several  data  points.  This  averaging  becomes  more  effective  as 
the  number  of  data  points  fitted  by  a  single  polynomial  increases.  However,  the  larger  the 
span  the  less  accurate  is  the  representation  of  the  data.  Hence  we  must  choose  a 
compromise  between  the  degree  of  smoothing  and  the  faithfulness  to  the  data. 


7 

of  Numerical  Differentiation.”  Comp.  ./.  3  (1960).  pp 

g 

^  R.  Simha.  ”Numerical  Differentiation  of  Equally  Spaced  and 

Not  Equally  Spaced  Experimental  Data.”  Ind.  Eng.  Chem.  Found  6  (1967 ).  pp  4 13-421. 

9 

P.  J.  Davis.  ”Interpolation  and  Approximation.”  Blaisdell  Publishing  Company  (1963).  p.  113. 
S.  Wold.  Spline  Functions  in  Data  Analysis.”  T echnometrics.  16  ( 1974 ).  pp  1-11. 
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A  polynomial  fitted  to  a  span  of  the  data  is  a  linear  combination  of  the  powers  of 
the  independent  variable.  This  procedure  can  be  refined  in  two  ways.  First,  we  may 
transform  the  data  and  then  fit  by  a  polynomial.  For  instance,  if  we  observe  y(t)  —  ce~^ 
then  log  y(t )  can  be  fitted  by  a  first  degree  polynomial.  Second,  we  can  fit  the  data  by  a 
linear  combination  of  functions  other  than  powers  of  the  independent  variable,  such  as 
exponentials,  trigonometric  functions  and  others.  Such  fits  are  called  linear  regression 
models  and  a  polynomial  fit  is  a  special  case  of  such  a  representation. 

Since  a  spline  function  can  be  represented  as  a  linear  combination  of  so-called  B- 
splines  an  approximation  by  a  spline  function  can  be  interpreted  as  a  refinement  of  the 
linear  regression  approach.  The  refinement  is  obtained  by  adding  a  condition  to  the 
approximation  criterion,  namely,  a  requirement  to  minimize  certain  combinations  of 
approximation  errors  and  the  norm  of  the  second  derivative  of  the  approximating 
function.  This  added  condition  is  included  by  the  following  considerations.  It  can  be  shown 
that,  of  all  the  functions  interpolating  the  data,  a  cubic  spline  has  second  derivatives  with 
the  minimal  L2  norm.  Consequently,  when  data  are  fitted  instead  of  interpolated,  it  is 
natural  to  select  a  function  )  that  represents  the  data  with  sufficient  accuracy  and  at  the 
same  time  minimizes 


i.e.,  to  minimize  J  subject  to  the  constraint 

,  (2.17) 


where  5  is  a  measure  of  desired  accuracy.  C.  M.  Reinsch^^  has  shown  that  the  solution  of 
this  minimization  problem  is  a  cubic  spline  with  the  nodes  •  This  function  is 

obtained  by  minimizing 


+  z  — 


S.  Anderssen  and  P.  Bloomfield,  "Numerical  Differentiation  Procedures  for  Non  Exact 
Data,”  Numer.  Math.  22  ( 1974),  pp  157-182. 

12 

C.  H.  Reinsch,  "Smoothing  by  Spline  Functions,"  Numer.  Math.  10  (1967),  pp  177-183. 
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where  2  is  a  slack  variable  and  is  a  Ugrange  multiplier.  By  the  method  of  calculus  of 
variations  it  can  be  shown  that  2=0.  Thus,  the  problem  is  to  minimize 


+  p 


(2.18) 


and  the  solution  to  this  problem  is  a  cubic  spline,  as  stated  above. 


j. 


KiitiULARIZATION 


Minimization  of  (2.18)  by  the  standard  methods  of  calculus  of  variations  expresses 
p  as  a  monotone  function  of  S.  Thus,  we  can  choose  S  and  determine  the  corresponding  d 
or  choo^  ;;  and  cornpute  S.  We  may  do  the  latter  since,  in  practice,  we  do  not  know  how 
clo^ly  the  tota  can  be  fitted.  With  p  -  Oconst  and  „  -  i/p  the  function,  /,  that  minimizes 
Aif  ,P)  IS  obtained  by  minimizing 


When  a  (and  hence  p)  is  specified  in  advance,  S  is  constant  (dependent  on  <,).  Hence  it 
can  be  omitted  from  the  expression  to  be  minimized.  Therefore  for  a  fixed  «  f  is 
obtained  by  minimizing  ’ 


5C/’,q:)-^52  [/’(^«) aj*  [/■''  (t)]  dt  .  (3.1) 

The  inte^al  in  this  relation  can  be  interpreted  as  the  square  of  a  seminorm  in  a  Banach 
space  of  functions  with  square  integrable  second  derivatives.  If  this  seminorm  is  replaced 
by  another  seminorm  involving  first  and  second  derivatives  we  obtain  /  by  minimizing 


S  \f'  {t)\ dt  +  j*  \f"  u)\dt 


(3.2) 


Here  the  integration  limits  have  been  changed  to  0  and  1  which 
rescaling  the  independent  variable  t. 


we  can  al\vays  do  by 


I 
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Let 


t  1 

/(?  )  «  /  g(T)  rfT  -  /  /»  a  -  T)g(T)  dr  , 

0  0 

where  /i(r)  is  the  Heaviside  step  function.  Then  fit)  -  git)  and /"(?)- g'(t).  Thus,  (3.2) 
can  be  written  as  follows: 


.  1  ^ 


rt-1 


1 

2 

1  r  1 

’ 

//»(?„—  r)giT)  dT—X„ 
0 

+ 

» 

/  [g  (t)]  dr+f 

o  '  0 

[g’  (t)]  dr 

(3.3) 


and  git )  is  obtained  by  minimizing  (3.3).  Thus,  replacing  the  seminorm 


r  r  1 

2 

/  \f"  (<) 
0 

dt 

by 


to 

r  12 

j  rf,+j 

0  0 

\f'it)\  dt 

reduces  the  spline  approximation  to  the  computation  of  a  derivative  by  the  regularization 
procedures  of  A.  N.  Tikhonov,^  provided  we  represent  his  functional 


mg,x) 


1 

S 

0 


1 

/ 

0 


hit  —  T)giT)  —xit) 


2 


dt 


in  a  discrete  form,  as  we  must  since  xit)  is  given  only  at  a  set  of  discrete  values  of  t  If 
we  replace  the  first  term  of  (3.3)  by  iV(g,x),we  get  Tikhonov’s  regularizing  functional: 
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By  equating  the  variation  of  C(^,a)  with  respect  to  g  to  zero  we  obtain 
necessary  condition  on  g  for  the  minimum  of  Cig,a): 


the  following 


1  1  1 

ffH9-T)/,(s-t)sM</Tae-f/,(e-,)x(9)de  +  agQ)-ag"(i)-o  .  (3.5) 


Transversality  conditions  yield: 


(3.5)  can  be  written  in  the  form: 


It  I 

-  gU)  +  ^//  gi9)  dOdT-^  /  x{e) 


d9 


Substituting  ?  =  1  in  (3.7)  gives: 


^''(1)-^(1)  +  — /  gi9)d9 
“  0 


We  differentiate  (3.7)  with  respect  to  t  and  obtain: 


^'"(O -^'(t)  +  — x(t) — -fg(9)d9  . 


a  a' 


(3.6) 


(3.7) 


(3.8) 


(3.9) 


When  t  =  1  (3.9)  yields: 


^"'(1) -^'(1)+— x(l)  —  —  r^(0)£/0  . 

a  « 0 

We  eliminate  the  integral  from  (3.8)  and  (3.10)  and  obtain: 


r'(l)+^''(l)-^(l)  +  lx(l)  . 

a 


(3.10) 


(3.11) 


In  view  of  (3.6),  substitution  of  t  =0  in  (3.9)  gives 
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(3.12) 


^'"(0)  -  -I  x(0)  . 
a 

By  differentiating  (3.9)  with  respect  to  t  we  get: 


)  -  g"{t )  +  ^gU )  -  ^x'(r )  . 
a  a 


(3.13) 


Thus,  the  optimizing  function  satisfies  differential  equation  (3.13)  with  the  boundary 

(3-12).  Eigenvalues  of  (3.13)  can  be  easily  expressed  in  terms 
of  a.  Denote  these  eigenvalues  by  ±x  and  ±At  .  Then  Green’s  function  of  (3.13)  is 


(3.14) 


The  general  solution  of  (3.13)  is: 


git)  -=  ^  +  A2,e>^  +  A^e  /  GU,r)  x'(t)  dr 

0 


We  integrate  the  last  integral  by  parts  and  obtain: 


g{.t)’-A,e^^ +A^e  ^  G,{t,r)  x{t)  dr  ,  (3  15) 

0 


where  GtU,r)  is  the  derivative  of  GU,r)  with  respect  to  t  and 


G,U,r)  =■  ^2  ^  ^2  ['^oshX(f  —  t)  —  cosh  (jlG  —  T)j  . 

The  comtants  of  integration  A^^  A^,  ^3,  and  A^  are  determined  by  the  boundary  conditions 
(  .  ),  (3.  ),  and  (3.12).  The  integral  in  (3.15)  can  be  approximated  by  a  numerical 

qu^ature  formula  such  as  trapezoidal  or  Simpson’s  rule  and  with  the  observed  values 
x„  ^x(t„)  .  Thus,  (3.15)  determines  an  approximation  of  the  derivative  g(t„)  for  a  given 
a.  With  the  exact  values  of  x(0,  g(t)  given  by  (3.15)  converges  to  x'(r)  as  a  — 0  When 
o^y  a  finite  number  of  values  x„  ^x(t„)  are  available  and  when  these  values  'contain 
o  servation  errors  the  test  approximation  of  x'(D  is  obtained  from  a  >  0  as  shown  by 
examples  of  ^e  artificial  data  with  exact  values  x'(t)  obtainable  by  direct  analytic 
differentiation.  In  the  case  of  real  data,  i.e.,  when  x'(t)  is  not  known,  the  "test"  value  of 
a  must  be  selected  by  an  empirical  criterion.  Wahba  proposes  a  cross-validation  procedure, 
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1 


by  which  approximation  git )  and  hence 

t 

/(?)-  /  gU)dt 
0 


is  determined  from  part  of  the  data,  say,  with  every  10th  point  omitted.  The  resulting 
values  of  fit)  for  omitted  points  are  compared  with  the  omitted  data.  The  value  of  alpha 
that  produces  the  least  RMS  of  these  differences  is  defined  as  the  "best"  a. 

An  alternative  procedure  was  proposed  by  R.  S.  Anderssen  and  P.  Bloomfield. It 
is  based  on  the  assumption  that  the  spectrum  of  observations  is  in  the  form 


b  +  b/ 


a[(w//)^  +  (w//)'^j 


where  I  =  —  ^d/at-  The  parameter  a  can  be  determined  by  approximating  the  spectrum 

.of  the  data  obtained  from  the  Fourier  transform.  This  process  is  based  on  the  assumption 
that  xit)  is  a  stochastic  process  and  that  spectra  of  x(/)  and  observation  errors  €„  do  not 
overlap. 


4.  AUTOREGRESSIVE  APPROACH 

As  stated  in  the  preceding  section,  the  regularization  procedure  is  a  refinement  of 
the  spline  approximatioj.  A  further  refinement  of  these  methods  is  provided  by  an 
autoregressive  approach.  This  approach  is  based  on  the  assumptions  which  imply  that  an 

approximating  derivative  satisfies  a  linear  homogeneous  differential  equation  with  constant 
coefficients. 


f=l 


For  any  a  this  equation  can  be  written  in  the  form: 


/«1  a  a 


(4.1) 


Since  git)  approximates  x'(t),  (4.1)  is  approximated  by 


13 

G.  Wahba,  ” Period  Splines  for  Spectral  Density  Estimation:  The  Use  of  Cross  Validation  for 
Determining  the  Degree  of  Smoothing,”  Commun.  in  Statistics  4  ( 1975),  pp  125-141. 
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(4.2) 


k 

z 


/-I 


-  — X'O) 
^  a 


which  is  a  generalization  of  (3.13).  Thus,  the  autoregressive  approach  is  a  refinement  of 
regularization  that  rarresponds  to  /i  -  4,  a,  -  U3  -  0,  a,  -  -1,  and  a,-l.  Instead  of  using  a 

^if  coefficients,  the  autoregressive  approach  determines  the 

order  k  and  the  coeffiaent  o,  from  the  observations.  Usually  the  data  allow  more  than  one 

choice  of  k  and  a/s.  An  empirical  criterion  is  selected  to  assign  relative  weights  to  everv 
*t  of  allowable  values  of  *  and  a/s.  These  weights  express  a  compromise  between  the 
faithfuln^  of  the  data  representation  and  the  stability  of  the  corresponding  autoregressive 
model.  The  approximation  of  the  derivative  is  the  weighted  average  of  these 
approximations.  Since  this  approach  selects  an  approximation  on  the  basis  of  the  data 
instead  of  an  a  priori  model  as  indicated  by  (3.13),  the  approximation  of  the  derivative 
obtain^  by  this  approach  is  more  accurate,  in  most  cases,  than  that  of  any  of  the  other 
methods  described  above. 
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